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PATENT 



SYSTEM AND METHOD FOR AUTOMATED 
CLASSIFICATION OF TEXT BY TIME SLICING 



TECHNICAL FIELD OF THE INVENTION 

5 

The present invention is directed, in general, to a system and 
method for the automated classification of text and, more 
specifically, to a system and method for the automated 
classification of the text of individual story segments that occur 
10 within video, audio and/or text broadcasts. Classification of the 
text of individual story segments may be used to categorize video 
segments so that the categorized video segments can bel selectively 
retrieved at later times. 



15 BACKGROUND OF THE INVENTION 

Computerized personal information retrieval systems exist for 
identifying and recording segments of radio or television 
broadcasts that contain topics that a user desires to record. The 
desired segments are usually identified based upon keywords input 
2 0 by the user. In a typical application, a computer system operates 
in the background to monitor the content of information from a 
source such as the Internet. The content selection is guided by 
the keywords provided by the user. When a match is found between 
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the keywords and the content of the monitored information, the 
information is stored for later replay and viewing by the user. 
Although the downloaded information may include links to audio or 
video clips that can also be downloaded by the user, the selection 
5 of the information for storage is based primarily on the frequency 
at which the keywords provided by the user appear in the text of 
the broadcast materials. 

A computerized personal information retrieval system that 
allows users to select and retrieve portions of radio or television 

10 programs for later playback usually meets three primary 
requirements. First, a system and method is usually available for 
parsing an incoming video signal into its visual, audio, and 
textual components. Second, a system and method is usually 
available for analyzing the content of the audio and/or textual 

15 components of the broadcast signal with respect to user input 
criteria and segmenting the components based upon content. Third, 
a system and method is usually available for integrating and 
storing program segments that match the user's requirements for 
later replay by the user. 

20 A system that meets these requirements is described in U.S. 

Patent Application Serial No. 09/006,657 filed January 13, 1998 by 
Dimitrova (a co- inventor of the present invention) entitled 
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"Multimedia Computer System with Story Segmentation Capability and 
Operating Program Therefor." U.S. Patent Application Serial No. 
09/006,657 is hereby incorporated by reference within this patent 
application for all purposes as if fully set forth herein. 
5 U.S. Patent Application Serial No. 09/006,657 describes a 

system and method that provides a set of models for recognizing a 
sequence of symbols, a matching model that identifies desired 
selection criterion, and a methodology for selecting and retrieving 
one or more video story segments or sequences based upon the 

10 selection criterion. 

U.S. Patent Application Serial No. 09/006,657 does not 
specifically address the problem that results when the methodology 
for segmenting the broadcast information into independent stories 
is primarily centered on visual content rather than video 

15 segmentation enhanced with audio and textual analysis. Currently, 
when video segmented stories are classified according to keywords, 
the analysis is based upon the assumption that the detection and 
required frequency of specified keywords in the segment provides an 
indication that the whole segment can be categorized by a single 

20 set of keywords. In actuality, there is a high probability that 
the frequency of appearance of specific keywords may change with 
time across a broadcast segment even when the video criteria for 
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story segmentation are satisfied. 

Therefore, video segmentation may result in one or more 
segments that have multiple stories within the section and that may 
be classified according to a single set of keywords. The resulting 
5 keywords that are selected to classify a segment with multiple 
stories may or may not be applicable to each story within the 
segment . 

There is therefore a need in the art for an improved system 
and method for identifying and segmenting broadcast information. In 

10 particular, there is need in the art for an improved system and 
method for identifying and segmenting broadcast information 
according to keywords. More particularly, there is a need in the 
art for an improved system and method for automated classification 
of the text of individual story segments that occur within a 

15 broadcast text over a period of time. 
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SUMMARY OF THE INVENTION 

To address the above -discussed deficiencies of the prior art, 
it is a primary object of the present invention to provide, for use 
5 in a multimedia broadcast system, an improved system and method for 
the automated classification of multimedia information according to 
audio and/or textual content over a period of time. Specifically, 
it is a primary object of the present invention to provide an 
improved system and method for categorizing video segments so that 

10 the video segments can be selectively retrieved at later times. 

The system of the present invention comprises a text 
classifier controller that reads text having one or more keywords 
contained within one or more story segments within the text. The 
text classifier controller identifies keywords within each line, 

15 and, in response to identifying at least one keyword within a line 
of text, classifies that line of text as a part of a story segment 
within the text. 

The text classifier controller identifies keyword transition 
points in the text where the number of detected keywords in a 

2 0 particular category of keywords decreases below a threshold number. 
The text classifier controller also identifies keyword transition 
points in the text where the number of detected keywords in a 
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particular category of keywords increases above a threshold number. 
The text classifier controller classifies story segments based on 
the location of the keyword transition points. 

It is an object of the present invention to provide an 
5 improved system and method for reading lines of text to identify 
keywords contained within the lines of text. 

It is also an object of the present invention to provide an 
improved system and method for classifying each line of text in a 
category of keywords that has the largest number of keywords in the 
10 line of text. 

It is another object of the present invention to provide an 
improved system and method for classifying story segments in a text 
by identifying and locating keyword transition points in the text. 

It is an additional object of the present invention to provide 
15 an improved system and method for classifying a story segment in a 
text by identifying and locating two adjacent keyword transition 
points in the text. 

It is also an object of the present invention to provide an 
improved system and method for classifying a story segment in a 
2 0 text when no keyword transition point is identified. 

The foregoing has outlined rather broadly the features and 
technical advantages of the present invention so that those skilled 
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in the art may better understand the detailed description of the 
invention that follows. Additional features and advantages of the 
invention will be described hereinafter that form the subject of 
the claims of the invention. Those skilled in the art should 
5 appreciate that they may readily use the conception and the 
specific embodiment disclosed as a basis for modifying or designing 
other structures for carrying out the same purposes of the present 
invention. Those skilled in the art should also realize that such 
equivalent constructions do not depart from the spirit and scope of 

10 the invention in its broadest form. 

Before undertaking the DETAILED DESCRIPTION, it may be 
advantageous to set forth definitions of certain words and phrases 
used throughout this patent document: the terms "include" and 
"comprise," as well as derivatives thereof, mean inclusion without 

15 limitation; the term "or," is inclusive, meaning and/or; the 
phrases "associated with" and "associated therewith," as well as 
derivatives thereof, may mean to include, be included within, 
interconnect with, contain, be contained within, connect to or 
with, couple to or with, be communicable with, cooperate with, 

2 0 interleave, juxtapose, be proximate to, be bound to or with, have, 
have a property of, or the like; and the term "controller" means 
any device, system or part thereof that controls at least one 
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operation, such a device may be implemented in hardware, firmware 
or software, or some combination of at least two of the same. It 
should be noted that the functionality associated with any 
particular controller may be centralized or distributed, whether 
5 locally or remotely. Definitions for certain words and phrases are 
provided throughout this patent document, those of ordinary skill 
in the art should understand that in many, if not most instances, 
such definitions apply to prior, as well as future uses of such 
defined words and phrases . 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present invention, 
and the advantages thereof, reference is now made to the following 
5 descriptions taken in conjunction with the accompanying drawings, 
wherein like numbers designate like objects, and in which: 

FIGURE 1 illustrates a high level block diagram of an 
exemplary multimedia computer system, according to one embodiment 
of the present invention; 
10 FIGURE 2 illustrates a high level block diagram of an 

exemplary video segmentation device, according to one embodiment of 
the present invention; 

FIGURE 3 illustrates a flow chart depicting the operation of 
an exemplary text classifier, according to one embodiment of the 
15 present invention; and 

FIGURE 4 illustrates a flow chart depicting the logical steps 
of an algorithm of the present invention for scanning each line of 
text in a textual segment to calculate the frequency of keywords 
within the line of text. 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURES 1 through 4, discussed below, and the various 
embodiments used to describe the principles of the present 
5 invention in this patent document are by way of illustration only 
and should not be construed in any way to limit the scope of the 
invention. Those skilled in the art will understand that the 
principles of the present invention may be implemented in any 
suitably arranged multimedia information system. 
M : 10 FIGURE 1 illustrates a high level block diagram 100 of an 

SP exemplary multimedia computer system 105 according to one 

5 SBfc embodiment of the present invention. Multimedia computer system 

j==? 105 comprises video story segmentation device 110, storage device 

jj; 12 0 and display device 13 0. According to one embodiment, video 

15 story segmentation device 110 may be a television set top box, 
storage device 12 0 may be a video cassette recorder (VCR) , and 
display device 130 may be a television set. 

In another embodiment, multimedia computer system 105 may be 
a multimedia-capable computer processing unit (CPU) equipped with 
2 0 a television tuner card and a read-write compact disc (CD-RW) 
drive. For this embodiment, the CPU and the television tuner card 
collectively constitute video story segmentation device 110, the 
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CD-RW drive constitutes storage device 12 0, and the computer 
display constitutes display device 13 0. In an alternate embodiment, 
one of a compact disc read-only-memory (CD-ROM) drive, a CD-RW 
drive, or a digital versatile disc (DVD) drive may provide the 
5 multimedia Video Input signal rather than a television broadcast or 
cable system. In another embodiment, the hard drive associated 
with the CPU may serve as storage device 120. Other configurations 
are also possible. For example, one embodiment may comprise a 
video story segmentation device 110 that is built into a video 

10 cassette recorder or a CD-RW drive. 

The term "video" may be interchanged with "multimedia . " In 
either case, the input signal comprises a time sequence of 
simultaneous video shots, audio segments, and text segments 
(e.g., closed captioned text, speech-to-text conversion, extracted 

15 video text) . The term "text" refers to a transcript of program 
materials. The term "text" may include, but is not limited to, 
transcripts of video text, transcripts of audio text, transcripts 
of closed captioned text, and generated transcripts of text. 

The Video Input signal may include time markers. 

2 0 Alternatively, time markers may be inserted into the Video Input by 
video story segmentation device 110. 

The term "multimedia signal" signifies a signal having a video 
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component and at least one other component (e.g., an audio 
component) . Multimedia signals may comprise video clips, video 
stream, video bitstream, video sequence, digital video signal, 
analog video signal, broadcast television signal, and similar 
5 signals, whether the signal is compressed or not. Although the 
following discussions refer to a multimedia signal which is a 
digital signal, the present invention is not restricted to 
digitized video or multimedia signals. 

FIGURE 2 illustrates a block diagram 200 of exemplary video 

10 story segmentation device 110 in greater detail, according to one 
embodiment of the present invention. Video story segmentation 
device 110 comprises video shot parsing device 210, audio parsing 
device 220, text parsing device 230, time extraction circuit 260, 
finite automaton (FA) library 24 0 with keywords library 245, event 

15 model recognition device 250, classification device 270 with text 
classifier 275, and classification storage device 280. 

FA library 240 and classification storage device 280 may be 
within a single memory device. The single memory device may be a 
non- volatile memory device such as a hard drive, flash memory, or 

2 0 programmable read-only memory (PROM) . The devices included in 
video story segmentation device 110 may be software modules for 
transforming a general purpose computer into a multimedia computer 
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system, where each of the modules resides in a program memory 
(i.e., a storage medium) until called by the system central 
processing unit (CPU) . A detailed description of the various 
devices included in video story segmentation device 110 will now be 
5 provided in terms of the corresponding software modules. 

Video shot parsing device 210, audio parsing device 22 0, and 
text parsing device 23 0 each function to separate their designated 
portion of the Video Input signal from the other components of the 
signal and to digitize these signals, as required, for transfer to 

10 the appropriate section of event model recognition device 250. In 
an advantageous embodiment, one of (or a combination of) video shot 
parsing device 210, audio parsing device 220, and text parsing 
device 23 0 may comprise a digital filter that is specifically 
configured for the frequency range of signals associated with the 

15 respective video, audio, or text based signals. Alternatively, one 
or more of video shot parsing device 230, audio parsing device 220, 
and text parsing device 210 may comprise software with algorithms 
for extracting the content of the associated Video Input signal 
component . 

2 0 Video shot parsing device 210 detects video cuts to produce a 

sequence of key frames and time stamps for these key frames that 
are provided as an output to event model recognition device 250. 
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Similarly, audio parsing device 220 emulates a speech recognition 
system that separates the extracted audio signal into discrete 
segment types, including voice, music, laughter, and silence. 
Audio parsing device 22 0 analyzes the extracted voice signals and 
5 detects and identifies speakers based upon this analysis. Audio 
parsing device 22 0 inserts time stamps into extracted and 
categorized audio data. Text parsing device 230 decodes the text 
associated with closed captioning and inserts time stamps with this 
text . 

10 Time extraction circuit 260 provides a clock signal which is 

used as the timing source for all devices within video story 
segmentation device 110. Time extraction circuit 260 provides time 
stamps that are inserted with separated visual, audio, and text 
information. The time stamps are used to identify signals that are 

15 received at the same or sequential points in time. 

Finite automaton (FA) library 240 provides storage for finite 
automaton event models that represent specific types of visual, 
audio, or text patterns for segmentation and labeling purposes. 
The concept of a finite automaton (FA) is similar to that used in 

2 0 compilers. The FA constructs models based on possible sequences of 
labels with a transition graph having nodes with arrows between the 
nodes that indicate allowable transitions from one node to another. 
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Sequential processes are modeled by paths through the transition 
graph from node to node along the arrows . Each path corresponds to 
a sequence of labels of successive nodes along the path. 

With respect to video signal story segmentation, each node of 
5 the FA represents an event with a symbolic label designating 
appropriate characteristics of the signal being analyzed. For 
example, FA library 240 may store labels representing a set of 
keyframes, a set of keywords, an audio feature designation for a 
time - interval , and similar characteristics of video, audio, and 

10 text data. In the selection of a path in the selection graph, each 
transition between nodes is based not only on the appearance of a 
particular label (symbol) , but on a collection of symbols extracted 
from the Video Input signal that represent text, video frame, and 
audio segments. As a result of using the FA, the labels of the 

15 nodes along the transition path provide a descriptive FA model for 
segments of the Video Input signal. 

FA library 24 0 may store FA data comprising symbols, nodes, 
and path information for pre-determined models that are 
representative of signal patterns within sequential visual, audio, 

20 and text signals. In addition, FA library 240 may provide storage 
for new FA data that may be created, for example, by event model 
recognition device 250 based upon characteristics of incoming data. 
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Keywords library 24 5 within FA library 240 provides storage 
for keywords that may be used as labels and for identifying 
keywords within text or digitized audio signals. Keyword library 
245 may provide a pre-determined collection of keywords (default 
5 keywords) that are representative of types of dialogue, such as 
dialogue associated with news programs, that may be present in a 
representative cross -sect ion of program segments that are monitored 
by a typical user. For example, keywords such as "murder" , 
"theft", "gunshot", "criminal" and "police" may be indicative of a 

10 story about crime. Similarly, keywords such as "Senate", "House 
of Representatives", "Washington", "Senate bill", "House bill" and 
"legislation" may be indicative of a political news story. Keyword 
library 245 may also store keywords that are user selected. 

Event model recognition device 250 synchronizes incoming 

15 video, audio, and text data from the respective video shot parsing 
device 210, audio parsing device 220, and text parsing device 230 
according to time extraction circuit 260. Event model recognition 
device 250 then compares the synchronized incoming data with known 
models from FA library 240. When event model recognition 

20 device 250 cannot find an event model from FA library 240 that 
matches particular time stamped segments of the incoming signals, 
event model recognition device 250 creates an event model that 
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conforms to the segment and labels and stores this model in FA 
library 240 for future use. 

After event model recognition device 250 has matched the 
incoming signals with an associated FA, the label of the FA and the 
5 keywords describing the FA characteristics may be transferred to 
classification device 270 for story analysis. The label assigned 
to a new FA model may be assigned by the user, or may be generated 
by multimedia computer system 105 using electronic programming 
guide (EPG) information, or may be generated by multimedia computer 

10 system 105 using a random label generator. 

Classification device 270 analyzes keyframes (frames without 
transitions) within the video shot with consideration to the 
determined FA characteristics. Using the FA characteristics and 
well known keyframe identification algorithms, classification 

15 device 270 identifies patterns within a video shot sequence for 
segmenting video signals into one or more groups of related 
sequences, where each related grouping is referred to as a story. 
Classification device 270 subsequently categorizes applicable video 
shots according to characteristic patterns associated with 

2 0 dialogue, news programming, sports, nature shots, and similar 
patterns . 

Classification device 270 applies reiterative processes in 
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which neighboring video shots are compared with the present 
keyframes to determine the presence of similar patterns 
(e.g., FAs) . During this latter processing or at some later time, 
classification device 270 may retrieve textual information 
5 (e.g., closed captioning) or audio information converted to text 
for additional comparison and segmentation purposes. Through the 
iterative comparison process, classification device 270 eventually 
assembles a sequence of key frames with audio and text that are 
associated as a related video segment (i.e., a story) . 

10 During initial classification or prior to storage, 

classification device 270 may further refine story segmentation of 
particular classes of stories. According to one embodiment of the 
present invention, classification device 270 may route stories with 
a news classification, for example, to exemplary text classifier 

15 275 for further analysis. 

Text classifier 275 comprises software that examines a story 
segment to identify and classify portions of the story segment 
according to keywords. Video story segmentation device 110 with 
text classifier 275 comprises a text classifier controller that is 

2 0 capable of carrying out the present invention. 

Text classifier 275 examines a story segment to identify and 
classify portions of the story segment according to keywords. As 
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previously described, each story segment is received from 
classification device 270 with an indication of the keywords that 
have been determined to provide a classification of the subject 
categories within the story segment. Text classifier 275 receives 
5 the identified keywords and scans each line of story text for the 
keywords. Text classifier 275 calculates the frequency of 
occurrence of each keyword in each line of text. Text classifier 
275 then classifies each line of text based on the frequency of 
occurrence of the detected keywords . 

10 Text classifier 275 records statistical data on the frequency 

of occurrence of the keywords with respect to the number and 
location of the text lines in the story segment. Text 
classifier 275 applies a search algorithm to the recorded 
statistics in order to detect possible keyword transition points 

15 where the frequency of occurrence of the keywords in the text 
significantly changes. 

For example, text classifier 275 determines that lines one 
through six in a story segment are classified in a crime story 
category. Line one through line six each contain at least one 

20 keyword from the crime story category. Text classifier 275 also 
determines that lines seven through fifteen are classified in a 
sports story category. Line seven through line fifteen each 



- 19 - 



PATENT 



contain at least one keyword from the sports story category but 
contain no keywords from the crime story category. Text classifier 
275 identifies a significant change in the keyword categories at a 
"keyword transition point " between line six and line seven. 
5 A "keyword transition point" is a point between two adjacent 

portions of text where the frequency of appearance of keywords 
detected in a keyword category for each line of text prior to the 
keyword transition point decreases significantly or decreases below 
a threshold number. A "keyword transition point" may also be a 

10 point between two adjacent portions of text where the frequency of 
appearance of keywords detected in a keyword category for each line 
of text prior to the keyword transition point increases 
significantly or increases above a threshold number. A threshold 
number may be chosen to be any integer and may be chosen to be as 

15 low as the integer one. 

In most instances a keyword transition point will mark a 
boundary where one story ends and another story begins. That is, 
when the number of keywords detected in a first keyword category 
for each line of text prior to a keyword transition point decreases 

2 0 below a first threshold number, then usually the number of keywords 
detected in a second keyword category for each line of text is 
increasing above a second threshold number for the second category. 
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The decrease in the number of keywords in the first keyword 
category represents the end of a first story. The increase in the 
number of keywords in the second keyword category represents the 
beginning of a second story. 
5 In some cases text classifier 275 will find a keyword 

transition point where there is a decrease in the number of 
keywords in a first keyword category but no increase in the number 
of keywords in any of the other possible keyword categories. Text 
classifier 2 75 then marks the keyword transition point as the end 

10 of the first story. The text after the keyword transition point is 
temporarily classified as "miscellaneous text" until text 
classifier 275 obtains sufficient additional information to 
determine an appropriate category for the miscellaneous text. 

A successive line threshold number may be used to minimize the 

15 likelihood of an erroneous determination of a keyword transition 
point. For example, assume that text classifier 275 finds five 
successive lines that have keywords that are classified as 
belonging to a sport category. Then the sixth line has a keyword 
that is classified as belonging to a crime category. Then the next 

2 0 five lines have keywords that are classified as belonging to the 
sports category. The likelihood is that some crime keyword was 
used in the sports story. For example, the sportswriter may have 
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said that "The referee's decision robbed the team of victory." The 
use of the crime keyword "robbed" in the sports story caused the 
line in which it appeared to be classified as a line of a crime 
story. 

5 In this case, setting the successive line threshold number to 

at least two would mean that there would have to be at least two 
successive lines having a keyword that is classified as belonging 
to a crime category before text classifier 275 would consider the 
possibility that a keyword transition point exists between the 

10 sports story portion and the crime story portion of the text. 

According to one embodiment, if the frequency of occurrence of 
keywords associated with the story classification is stable or 
rising, text classifier 275 categorizes that portion of the text as 
being appropriately classified by the detected keywords. 

15 However, when text classifier 275 detects a keyword transition 

point where the number of detected keywords decreases below a 
threshold number, text classifier 275 may identify the keyword 
transition point as the possible end of one story and the possible 
beginning of another story. Text classifier 275 then identifies a 

2 0 possible new story segment beginning with the possible keyword 
transition point through the end of the original story segment. 
The story segment from the possible keyword transition point 



- 22 - 



PATENT 



through the end of the original story segment is referred to as a 
partial story segment. 

Text classifier 2 75 examines the partial story segment 
according to the frequency of occurrence of the keywords. 
5 According to an advantageous embodiment, if text classifier 2 75 
finds an insufficient number of keywords to satisfy a minimum 
threshold number, text classifier 2 75 may determine 1) that the 
partial story segment is a transition segment from the original 
story, and 2) depending on the lines that follow the partial story 

10 segment may or may not determine that the partial story segment is 
to remain a part of the original story. 

According to another advantageous embodiment, text classifier 
275 may determine that the partial story segment also has a keyword 
transition point where the number of detected keywords associated 

15 with the current portion of the partial story segment decreases 
below a threshold number. Text classifier 275 may then classify 
the current portion of the partial story segment as a second story. 
Text classifier 275 then identifies another possible new story 
segment beginning with the keyword transition point through the end 

2 0 of the original story segment. 

According to another advantageous embodiment, text classifier 
275 may determine that the partial story segment also has a keyword 
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transition point where the number of detected keywords not 
associated with the current portion of the partial story segment 
increases above a threshold number. Text classifier 275 may then 
classify the current portion of the partial story segment as a 
5 second story. Text classifier 275 then identifies another possible 
new story segment beginning with the keyword transition point 
through the end of the original story segment . 

After text classifier 275 completes the examination of the 
original story segment with respect to one or more categories of 

10 keyword classifications and finds no additional keyword transition 
points, text classifier 275 examines the results of the keyword 
analysis and the elapsed time between keyword classification 
determinations to make a final determination of the number of 
individual stories within the original story segment. Text 

15 classifier 275 groups the new story segments, if any, according to 
keyword classification. Text classifier 275 then begins processing 
the next received story segment using the same procedure. 

In an advantageous embodiment, text classifier 2 75 may apply 
a keyword algorithm that identifies the end of one story and the 

2 0 beginning of another story as the last text line that has an 
occurrence of the indicated keyword. In another embodiment, text 
classifier 275 may apply algorithms to further analyze story 
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segments based upon the time duration of a particular classified 
segment. Within selected time thresholds, text classifier 275 may 
then classify a particular segment as being an advertisement, 
a public announcement, a teaser for an upcoming news story, or a 
5 similar type of story with a particular elapsed time interval. 

Classification storage device 280 stores classified video 
segments from classification device 270 for immediate or delayed 
transfer to storage device 12 0 and eventual display by display 
device 13 0, as specified by the user. 

10 FIGURE 3 illustrates flow chart 300 depicting the operation of 

text classifier 275, according to one embodiment of the present 
invention. During the operation of video story segmentation device 
110, text classifier 275 receives a classified story and its 
associated keyword list from event model recognition device 250 and 

15 classification device 270 (process step 305) . Text classifier 275 
then scans each line of text in the story for the keyword (s) that 
are included within the keyword list. Text classifier 275 then 
applies an algorithm (illustrated in FIGURE 4) to determine the 
frequency of occurrence of the keyword (s) for each line of text. 

2 0 This algorithm will be described more fully below. The algorithm 
classifies each line of text within the particular story based upon 
the number of keyword (s) found in each individual line of text 
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(process step 310) . 

Text classifier 275 then examines the keyword statistics that 
result from the scanning process to detect one or more keyword 
transition points in the story. As previously mentioned, a keyword 
5 transition point is a point where the frequency of appearance of 
keyword (s) from the keyword list decreases significantly or 
decreases below a threshold number (process step 315) . 

If at least one keyword transition point is detected in the 
story (process step 320) , text classifier 275 classifies the 
10 portion of the story segment with the most frequent occurrence of 
keywords from the keyword list according to the category of the 
keyword list and temporarily stores the classified story segment 
(process step 330) . 

After text classifier 275 classifies and temporarily stores a 
15 first portion of the story (process step 330) , text classifier 275 
causes classification device 270 to transfer the remaining 
portion (s) of the original story to event model recognition device 
2 50 for determination of a new keyword list that characterizes the 
remaining story portion(s) as one story (process step 335). 
2 0 Text classifier 2 75 monitors the status of event model 

recognition device 250 until a keyword list is available for the 
remaining story portion (s) (process step 340) . Text classifier 275 
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then starts the text classification process over again and receives 
the story portion (s) with associated keyword list from event model 
recognition device 250 and classification device 270, as previously 
described (process step 305) . 
5 The described process continues until text classifier 275 

detects no keyword transition points within the story segment being 
classified (at which point text classifier 275 has detected the end 
of the text) (process step 320) . When no keyword transition point 
is detected (and the end of the text has been detected) (process 

10 step 320) , text classifier 275 combines the story portion being 
processed and its keyword list with any other story portion (s) and 
their keyword list(s) to create an integrated story and keyword 
list. Text classifier 275 then transfers the resultant story 
portion (s) and keyword list (s) to classification storage device 280 

15 (process step 325) . Video segmentation device 110 then continues 
its normal operation. 

FIGURE 4 illustrates flow chart 400 depicting the operation of 
an algorithm for scanning each line of text as described in process 
step 310. An array of computer memory locations having dimensions 

2 0 N by J is prepared to receive information concerning the identity 
and location of keywords that are detected in the text. The name 
of the N by J array is Hit Counter (N, J) . The letter N denotes the 
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number of a line of text. The letter J denotes the number of a 
keyword in the list of keywords. At the beginning of the algorithm 
each memory location in Hit Counter (N,J) is set equal to zero 
(process step 405) . 
5 An array of computer memory locations having dimensions N by 

M is prepared to receive N lines of text where each line of text is 
no longer than M characters. N is an integer index that 
sequentially points to each line in the text. The name of the 
variable that points to one line of text is Line (N) . When N 

10 equals one (N = l),then Line (N) points to the first line in the 
text. When N equals two (N = 2) , then Line (N) points to the second 
line in the text, and so on. 

In each Line (N) of text the individual words in Line (N) are 
sequentially identified as Word (I) . I is an integer index that 

15 sequentially points to each word in Line (N) of text. That is, when 
I equals one (1=1), then Word (I) points to the first word in 
Line (N) . When I equals two (I = 2) , then Word (I) points to the 
second word in Line (N) , and so on. 

An array of computer memory locations having dimensions J by 

2 0 K is prepared to receive J keywords where each keyword in the array 
is no longer than K characters. The keyword list resides in the J 
by K array of computer memory locations. J is an integer index 
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that sequentially points to each keyword in the keyword list. The 
name of the variable that points to one keyword is Keyword (J) . 
When J equals one (J = l),then Keyword (I) points to the first 
keyword in the keyword list, and so on. 
5 The number N is set equal to one (1) to cause Line (N) to 

point the first line of text (process step 405) . The number I is 
set equal to one (1) to cause Word (I) to point to the first word 
in Line (N) (process step 4 05) . The number J is set equal to one 
(1) to cause Keyword (J) to point to the first keyword in the 

10 keyword list (process step 405) . 

The text in Line (N) is then retrieved into computer memory 
(process step 410) . The first word Word (I) in Line (N) is then 
retrieved into computer memory (process step 415) . Keyword (J) is 
then retrieved into computer memory (process step 42 0) . 

15 A determination is then made as to whether Word (I) is equal to 
Keyword (J) (decision step 425) . 

If Word (I) is not equal to Keyword (J) , then a determination 
is made as to whether Keyword (J) is the last keyword in the 
keyword list (decision step 430) . If Keyword (J) is not the last 

2 0 keyword in the keyword list, then the integer value of J is 
incremented by one to point Keyword (J) to the next keyword in the 
keyword list (process step 435) . The next keyword in the keyword 
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list is retrieved into computer memory (process step 420) . Control 
then passes to decision step 425 for further processing. 

When decision step 425 finds that Word (I) equals Keyword (J) , 
then the appropriate array location in Hit Counter (N, J) is 
5 incremented to record one "hit" of Keyword (J) in Line (N) (process 
step 450) . Control then passes to decision step 445 to determine 
whether Word (I) is the last word in Line (N) . Control also passes 
to decision step 445 from decision step 43 0 when it is determined 
that a particular Keyword (J) is the last keyword in the keyword 
10 list. 

If Word (I) is not the last word of Line (N) , then the integer 
value I is incremented by one to point Word (I) to the next word in 
Line (N) (process step 450) . In addition, J is set equal to one to 
point Keyword (J) to the first keyword in the keyword list (process 

15 step 450) . Control then passes to process step 415 to get the next 
Word (I) of Line (N) for further processing. 

If Word (I) is the last word of Line (N) , then Line (N) is 
classified in the category that has the largest number of keyword 
detections (or "hits") in Hit Counter (N, J) (process step 455) . 

2 0 A determination is then made as to whether Line (N) is the last 
line in the text (decision step 460) . If Line (N) is the last line 
in the text, then the algorithm described in flow chart 4 00 ends 
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and control of the program passes to process step 315. If Line (N) 
is not the last line in the text, then the integer value N is 
incremented by one to point Line (N) to the next line of text 
(process step 470) . In addition, I is set equal to one to point 
5 Word (I) to the first word in Line (N) (process step 465) . Control 
then passes to process step 410 to get the next Line (N) of text 
for further processing. 

Although selected keywords are often found to occur in the 
most recent portion of a story segment, this may not always be the 

10 case. It is possible that an original story segment may be 
classified according to keywords that appear in the latter portion, 
in the middle portion, or in some other portion of the story 
segment. It should be understood that text classifier 275 
comprises algorithms that are capable of identifying and isolating 

15 stories within the original segment, regardless of the location of 
the stories within the original segment. It should be further 
understood that text classifier 275 in association with video story 
segmentation device 110 is capable of isolating remaining stories 
within the original story segment regardless of the time interval 

2 0 in which the remaining stories appear. 

In an alternate embodiment, text classifier 2 75 may be used in 
conjunction with the initial video and audio classification 
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algorithms. In this alternate embodiment, text classifier 275, 
working in conjunction with other portions of classification device 
270, may classify segments as they occur according to the keyword 
algorithms that classify stories based upon the frequency of 
5 appearance of keywords over a period of time or within lines of 
text. In this alternate embodiment, text classifier 2 75 may also 
re-examine adjacent story segments to determine the possible 
presence of text lines which may be associated with the segment 
being presently classified. 

10 In an alternate embodiment, FA library 24 0 and event model 

recognition device 250 may work within video story segmentation 
device 110 to separate and identify story segments based only on 
the characteristics of the audio and video components of the Video 
Input signal (as opposed to also using the text components of the 

15 Video Input signal) . The audio components may be converted to text 
to separate and identify story segments but the text that comes 
directly from text parsing device 230 (e.g., closed captioned text, 
speech-to-text conversion, extracted video text) is not used to 
separate and identify story segments. 

20 Classification device 270 receives each resultant story 

segment with its associated text. The associated text may include 
text converted from audio and text directly from text parsing 
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device 230. Text classifier 275 then further segments the story 
segment based upon keywords from keyword library 245. Text 
classifier 275 performs a line by line analysis as illustrated in 
FIGURE 4. When each line of text is classified, text classifier 
5 275 detects keyword transition points and classifies portions of 
the story segment based upon the resultant keyword list for the 
particular segment. Text classifier 275 then transfers the 
resultant story segment (s) and associated keyword list(s) to 
classification storage device 280. 
10 Although the present invention has been described in detail, 

those skilled in the art should understand that they can make 
various changes, substitutions and alterations herein without 
departing from the spirit and scope of the invention in its 
broadest form . 
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WHAT IS CLAIMED IS: 

1 1 . For use in an information processing system, an apparatus 

2 for automatically classifying text comprising: 

3 a text classifier controller capable of reading text 

4 having at least one keyword contained within at least one story 

5 segment within said text, and capable of identifying keywords 

6 within each line of said text, and, in response to identifying at 

7 least one of said keywords within a line of text, classifying said 

8 line of text as a part of said at least one story segment within 

9 said text. 

1 2 . The apparatus as set forth in Claim 1 wherein said text 

2 classifier controller is capable of sequentially comparing first 

3 and second lines of text to compare the number of keywords detected 

4 for each first line of text with the number of keywords detected 

5 for each second line of text, and capable of identifying a keyword 

6 transition point between two adjacent portions of text where the 

7 number of keywords detected in a keyword category for each line of 

8 text prior to said keyword transition point decreases below a 

9 threshold number. 
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1 3 . The apparatus as set forth in Claim 2 wherein said text 

2 classifier controller is capable of classifying text between the 

3 beginning of the text and a first keyword transition point as one 

4 story segment of the text when said text classifier controller 

5 identifies a first keyword transition point. 

1 4 . The apparatus as set forth in Claim 2 wherein said text 

2 classifier controller is capable of classifying text between a 

3 first keyword transition point and a second keyword transition 

4 point as one story segment of the text when said text classifier 

5 controller identifies a first keyword transition point and a second 

6 keyword transition point. 

1 5 . The apparatus as set forth in Claim 1 wherein said text 

2 classifier controller is capable of sequentially comparing first 

3 and second lines of text to compare the number of keywords detected 

4 for each first line of text with the number of keywords detected 

5 for each second line of text, and capable of identifying a keyword 

6 transition point between two adjacent portions of text where the 

7 number of keywords detected in a keyword category for each line of 

8 text prior to said keyword transition point increases above a 

9 threshold number. 
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1 6 . The apparatus as set forth in Claim 1 wherein said text 

2 classifier controller comprises an algorithm for reading lines of 

3 text to identify keywords contained within said lines of text, 

4 wherein said algorithm classifies each line of text in a keyword 

5 category that has the largest number of keywords in said line of 

6 text . 

1 7 . An information processing system of the type comprising 

2 a video story segmentation device, a keyword library and a 

3 classification device wherein said information processing system 

4 comprises : 

5 a text classifier controller capable of reading text having at 

6 least one keyword contained within at least one story segment 

7 within said text, and capable of identifying keywords within each 

8 line of said text, and, in response to identifying at least one of 

9 said keywords within a line of text, classifying said line of text 
10 as a part of said at least one story segment within said text. 
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1 8 . An information processing system as set forth in Claim 7 

2 wherein said text classifier controller is capable of sequentially 

3 comparing first and second lines of text to compare the number of 

4 keywords detected for each first line of text with the number of 

5 keywords detected for each second line of text, and capable of 

6 identifying a keyword transition point between two adjacent 

7 portions of text where the number of keywords detected in a keyword 

8 category for each line of text prior to said keyword transition 

9 point decreases below a threshold number. 

1 9 . An information processing system as set forth in Claim 8 

2 wherein said text classifier controller is capable of classifying 

3 text between the beginning of the text and a first keyword 

4 transition point as one story segment of the text when said text 

5 classifier controller identifies a first keyword transition point. 

1 10. An information processing system as set forth in Claim 8 

2 wherein said text classifier controller is capable of classifying 

3 text between a first keyword transition point and a second keyword 

4 transition point as one story segment of the text when said text 

5 classifier controller identifies a first keyword transition point 

6 and a second keyword transition point. 
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1 11. An information processing system as set forth in Claim 7 

2 wherein said text classifier controller is capable of sequentially 

3 comparing first and second lines of text to compare the number of 

4 keywords detected for each first line of text with the number of 

5 keywords detected for each second line of text, and capable of 

6 identifying a keyword transition point between two adjacent 

7 portions of text where the number of keywords detected in a keyword 

8 category for each line of text prior to said keyword transition 

9 point increases above a threshold number. 

1 12 . An information processing system as set forth in Claim 7 

2 wherein said text classifier controller comprises an algorithm for 

3 reading lines of text to identify keywords contained within said 

4 lines of text, wherein said algorithm classifies each line of text 

5 in a keyword category that has the largest number of keywords in 

6 said line of text. 
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1 13 . For use in an information processing system, a method of 

2 automatically classifying text comprising the steps of: 

3 reading text having at least one keyword contained within 

4 at least one story segment within said text; 

5 identifying keywords within each line of said text; and 

6 in response to identifying at least one of said keywords 

7 within a line of text, classifying said line of text as a part of 

8 said at least one story segment within said text. 

1 14. The method as set forth in Claim 13 further comprising 

2 the steps of : 

3 sequentially comparing first and second lines of text to 

4 compare the number of keywords detected for each first line of text 

5 with the number of keywords detected for each second line of text; 

6 and 

7 identifying a keyword transition point between two 

8 adjacent portions of text where the number of keywords detected in 

9 a keyword category for each line of text prior to said keyword 
10 transition point decreases below a threshold number. 
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1 15. The method as set forth in Claim 14 further comprising 

2 the step of : 

3 classifying text between the beginning of the text and a 

4 first keyword transition point as one story segment of the text 

5 when said text classifier controller identifies a first keyword 

6 transition point. 

1 16. The method as set forth in Claim 14 further comprising 

2 the step of : 

3 classifying text between a first keyword transition point 

4 and a second keyword transition point as one story segment of the 

5 text when said text classifier controller identifies a first 

6 keyword transition point and a second keyword transition point. 
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1 17. The method as set forth in Claim 13 further comprising 

2 the steps of : 

3 sequentially comparing first and second lines of text to 

4 compare the number of keywords detected for each first line of text 

5 with the number of keywords detected for each second line of text; 

6 and 

7 identifying a keyword transition point between two 

8 adjacent portions of text where the number of keywords detected in 

9 a keyword category for each line of text prior to said keyword 
10 transition point increases above a threshold number. 
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1 18. The method as set forth in Claim 13 further comprising 

2 the steps of : 

3 sequentially comparing first and second lines of text to 

4 compare the number of keywords detected for each first line of text 

5 with the number of keywords detected for each second line of text; 

6 and 

7 identifying a plurality of keyword transition points in 

8 said text between adj acent portions of text where the number of 

9 keywords detected in a keyword category for each line of text prior 

10 to each of said keyword transition points decreases below a 

11 threshold number; and 

12 classifying the text between each two adjacent keyword 

13 transition points of said plurality of keyword transition points as 

14 a story segment of the text . 
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1 19. For use in a computerized information processing system, 

2 computer-executable instructions stored on a computer-readable 

3 storage medium for automatically classifying text, the computer- 

4 executable instructions comprising the steps of: 

5 reading text having at least one keyword contained within 

6 at least one story segment within said text; 

7 identifying keywords within each line of said text; and 

8 in response to identifying at least one of said keywords 

9 within a line of text, classifying said line of text as a part of 
10 said at least one story segment within said text. 

1 20. The computer-executable instructions stored on a 

2 computer-readable storage medium as set forth in Claim 19, the 

3 computer-executable instructions further comprising the steps of: 

4 sequentially comparing first and second lines of text to 

5 compare the number of keywords detected for each first line of text 

6 with the number of keywords detected for each second line of text; 

7 and 

8 identifying a keyword transition point between two 

9 adjacent portions of text where the number of keywords detected in 

10 a keyword category for each line of text prior to said keyword 

11 transition point decreases below a threshold number. 
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1 21. The computer-executable instructions stored on a 

2 computer- readable storage medium as set forth in Claim 20, the 

3 computer- executable instructions further comprising the step of: 

4 classifying text between the beginning of the text and a 

5 first keyword transition point as one story segment of the text 

6 when said text classifier controller identifies a first keyword 

7 transition point. 

1 22. The computer-executable instructions stored on a 

2 computer-readable storage medium as set forth in Claim 20, the 

3 computer-executable instructions further comprising the step of: 

4 classifying text between a first keyword transition point 

5 and a second keyword transition point as one story segment of the 

6 text when said text classifier controller identifies a first 

7 keyword transition point and a second keyword transition point. 
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1 23. The computer-executable instructions stored on a 

2 computer-readable storage medium as set forth in Claim 19, the 

3 computer-executable instructions further comprising the step of: 

4 sequentially comparing first and second lines of text to 

5 compare the number of keywords detected for each first line of text 

6 with the number of keywords detected for each second line of text; 

7 and 

8 identifying a keyword transition point between two 

9 adjacent portions of text where the number of keywords detected in 

10 a keyword category for each line of text prior to said keyword 

11 transition point increases above a threshold number. 
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1 24. The computer- executable instructions stored on a 

2 computer- readable storage medium as set forth in Claim 19, the 

3 computer-executable instructions further comprising the steps of: 

4 sequentially comparing first and second lines of text to 

5 compare the number of keywords detected for each first line of text 

6 with the number of keywords detected for each second line of text; 

7 and 

8 identifying a plurality of keyword transition points in 

9 said text between adjacent portions of text where the number of 

10 keywords detected in a keyword category for each line of text prior 

11 to each of said keyword transition points decreases below a 

12 threshold number; and 

13 classifying the text between each two adjacent keyword 

14 transition points of said plurality of keyword transition points as 

15 a story segment of the text. 
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SYSTEM AND METHOD FOR AUTOMATED 
CLASSIFICATION OF TEXT BY TIME SLICING 

ABSTRACT OF THE DISCLOSURE 

5 For use in an information processing system, there is 

disclosed a system and method for automatically classifying text. 
The system comprises a text classifier controller that reads text 
having one or more keywords contained within one or more story 
segments within the text. The text classifier controller 

10 identifies keywords within each line, and, in response to 
identifying at least one keyword within a line of text, classifies 
that line of text as a part of a story segment within the text. The 
text classifier controller also identifies keyword transition 
points in the text where the number of detected keywords in a 

15 particular category of keywords decreases below a threshold number. 
The text classifier controller also identifies keyword transition 
points in the text where the number of detected keywords in a 
particular category of keywords increases above a threshold number. 
The text classifier controller classifies story segments based on 

20 the location of the keyword transition points. 
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